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D Abstract 

' In this paper we prove estimates on the behaviour of the Kolmo- 

0^ . gorov-Sinai entropy relative to a partition for randomly perturbed dy- 

04 ' namical systems. Our estimates use the entropy for the unperturbed 

system and are obtained using the notion of Algorithmic Information 
' Content. The main result is an extension of known results to study 

Q , time series obtained by the observation of real systems. 

d ■ 1 Introduction 

In this paper we are interested in studying the behaviour of the Kolmogorov- 
Sinai (KS) entropy for randomly perturbed dynamical systems. As it has 
^ ' been proved in TT for a typical situation, the KS entropy of a dynamical 

■ system perturbed by a sequence of random variables is infinite for all a > 0; 

, here a is a real parameter measuring the size of the perturbation. This result 

I shows that the KS entropy is not the right quantity to look at in randomly 

' perturbed dynamical systems. In it has been suggested to study the KS 

. entropy relative to a partition. 

We study the perturbation of a dynamical system (X, fi, f) by some 
noise. By noise we mean a discrete stochastic processes of independent 
and identically distributed (i.i.d.) random variables {u^nlngN defined on a 
probability space {W, V, P) with values on the interval [—(J, <7], for a positive 
real number a. The number a is the parameter that measures the size of 
^ ■ the perturbation on the original system. 

We also assume that for all a there is an ergodic invariant probabil- 
ity measure ^jl^ on the perturbed system (X, /o-) such that the probability 
measure /U is the weak limit of the sequence ^o- as o" — > (for the formal 
definition of //o- see Section^. 

In it has been proved that, under these hypothesis, it holds 

limsup Z) < h^if, Z) < h^if) (1) 

0-^0+ 
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where Z) and hfj_^(f^, Z) are the KS entropies relative to the partition 
Z of the original and the perturbed system respectively, and by hfj_{f) we 
denote the KS entropy of the unperturbed dynamical system. The equality 
holds for certain hyperbolic systems (17,). 

In equation it is used the KS entropy relative to a fixed partition, 
avoiding the problem of having infinite KS entropy, and it is performed the 
limit for the size of the perturbation going to zero. This result shows that 
for an unperturbed dynamical system, the KS entropy relative to a partition 
is well approximated by the same quantity for the perturbed system, when 
the perturbation is small enough. 

Unfortunately when we want to study time series obtained by measure- 
ments of real systems, it is impossible to know how small is the perturbation 
to the original system. Hence it is impossible to perform the limit in equa- 
tion (HJ. 

Real systems are observed using a symbolic representation of the mea- 
surements. That is the phase space is divided into a finite number of sets, 
and we study the time series given by the symbols of the sets of the partition 
visited by the system as time grows. 

The main idea to study real systems is to look at the system using 
different scales of observation, which corresponds to use different sizes for 
the partition of the phase space. When the partition is coarse the effects of 
the noise are negligible, whereas, when the partition is very fine, then the 
effects of the noise hide the dynamics of the original system. Then the right 
approach to randomly perturbed systems is to study the behaviour of the 
KS entropy while the partition is refined and the noise is fixed. 

In this paper we prove that, when the diameter of the partition is big 
with respect to the size of the noise, the KS entropy of the perturbed system 
is a good approximation for the unperturbed KS entropy. Our main result is 
contained in equations H14|) and (jlSf) . which in particular imply equation ((H). 
Moreover we obtain precise estimates for the effects of the noise, in order 
to distinguish between the perturbation and the dynamics of the system, a 
very important problem in the analysis of time series (^). We remark that 
the main extension of the result contained in equation is its applicability 
to time series obtained by observations of real systems. 

To apply our method we have to face the problem of the measurement 
of the KS entropy in real systems (that is the KS entropy of observed time 
series). This is an important problem (for a review see for example PP), and 
many papers have been dedicated to the approximation of the KS entropy 
by different indicators. We recall the approach of Grassberger and Procaccia 
jl5j based on the generalized Renyi entropy, and that of Cohen and Procaccia 
jllj based on the correlation integral for a time series (16,). In this paper 
we use a method introduced in |^ and [1], which is related to the notion of 
Algorithmic Information Content (AIC) (see Section IHl for the definitions). 

Moreover, when applying theoretical methods to real time series, it is 
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always questionable whether all the hypothesis needed for the methods are 
verified by the single observations we study. This is this case, for example, 
for ergodicity that we suppose to be verified. 

In the following section, we recall some basic definitions and results re- 
lated to the KS entropy and its computation for dynamical systems and 
purely random systems. In Sections |1] and we apply the method of the 
AIC to randomly perturbed systems, and finally in Section IS] we show some 
numerical experiments performed on the logistic map at the chaotic param- 
eter A = 4. 

2 Basic definitions and results 

We now briefly recall the basic definitions we need. Let (X, B, /x) be a 
probability space, where X is a compact metric space and B is the Borel a - 
algebra. In this paper we restrict our attention on one dimensional spaces 
X, but we believe that the techniques used can be generalized to higher 
dimensions. Let f : X ^ X he a continuous map, invariant with respect to 
/X and ergodic. Given a finite measurable partition Z = {/j}i=i,...,iv of the 
space X, the entropy H^{Z) of the partition is defined as 

N 

H^{Z) = -Y, log(M^.)) (2) 

i=l 

Let f^^Z be the partition given by the counter images f^^h- Then let 

Z„ = ZV/-^ZV---V/-"+^Z (3) 
be the partition given by the sets of the form 

n r^h, n • • • n r^^^h^., 

varying Ij. among all the sets of Z. Then the Kolmogorov- Sinai entropy 
h^{f, Z) relative to the partition Z is defined as the limit 

h^{f,Z)= lim -H^iZn) (4) 

The Kolmogorov- Sinai entropy h^[f) of the dynamical system (X, ^,/) is 
defined as 

hfiif) = sup{/i^(/, Z) / Z finite partition} (5) 

Moreover for the explicit computation of the KS entropy, the Kolmogorov- 
Sinai theorem says that there are some special partitions, called generating, 
for which the supremum among all the partitions is realized. Hence it is 
enough to compute the KS entropy relative to a generating partition to 
obtain the KS entropy of the system. The existence of a generating partition 
for dynamical systems is given by the following theorem: 
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Theorem 2.1 (Krieger Generator Theorem jl9p . Let {X,iJ,,f) be an 
ergodic dynamical system on a Lebesgue space X, such that the probability 
measure fi is invariant and h^{X,f) < oo. Then there is a finite generating 
partition for f. 

Let e be the diameter of a uniform partition Z of the space X, and simply 
denote by hf^{e) the KS entropy h^{f,Z) relative to the partition Z. This 
notation has the aim to enhance the role of the diameter of the partition 
considered. The quantity /i^(e) is also called the e-entropy of the dynamical 
system (see |14j for a review). The Krieger Generator Theorem implies that, 
if h^{f) is finite, there exists eo such that /i^(e) = hfj,{f) for all e < eo. 

It is also possible to establish the behaviour of the e-entropy for discrete 
stochastic processes of i.i.d. random variables. 

Theorem 2.2 (Gaspard-Wang |14p . Let {wn}neN be a discrete stochas- 
tic process of i.i.d. random variables with values on the interval [0, 1]. Then 
as e ^ it holds 



where the entropy is computed with respect to the invariant probability mea- 
sure of the system ( the induced measure ) absolutely continuous with respect 
to the Lebesgue measure. 

When dealing with randomly perturbed systems, it has been shown using 
numerical experiments ([14j.pT]) that the expected behaviour is 



where a denotes the standard deviation of the noise (that is the square root 
of the variance of the random variables Wn)- The same result is expected 
for the generalized Renyi entropy with q = 2 (\2,2\]. 

We recall that for randomly perturbed systems the KS entropy h^^{ffj) 
is infinite, hence the Krieger Generator Theorem is not applicable to the 
system (X, /o-). But we assume that the unperturbed system (X,/) has 
finite KS entropy h^{f) with generating partition of diameter eg. Hence 
h^{e) = h^{f) for all e < cq. This implies that if the size a of the noise 
is small with respect to eo, approximating /i.^(e) by hfj_^{e) we find a good 
approximation to the KS entropy of the unperturbed system. 

In the following we prove © using the Algorithmic Information Con- 
tent (AIC), briefly described in the following section. We remark that the 
behaviour of equation © is of great importance since it would give us a 
method to estimate the size of the random perturbation on the determinis- 
tic dynamics of the system we are observing. 
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3 The algorithmic information content 



Given a finite alphabet A, let be the set of all words on the alphabet A 
of length n. The intuitive meaning of quantity of information contained in 
a finite string s € A^ is the length of the smallest message from which you 
can reconstruct s on some machine. Thus, formally, the information / is a 
function 

I -.A* = \J A"" 

nSN 

on the set of finite strings on a finite alphabet A which takes values in the 
set of natural numbers. 

One of the most important information function is the Algorithmic In- 
formation Content (AIC ). In order to define it, it is necessary to define the 
notion of partial recursive function. We limit ourselves to give an intuitive 
idea which is very close to the formal definition. We can consider a partial 
recursive function as a computer C which takes a program P (namely a 
binary string) as an input, performs some computations, and gives a string 
s = C{P), written on the given alphabet A, as an output. The AIC of a 
string s is defined as the shortest binary program P which gives s as its 
output, namely 

AIC{s, C) = min{|P| : C{P) = s} (7) 

We require that our computer is a universal computing machine. Roughly 
speaking, a computing machine is called universal if it can simulate any 
other machine. In particular every real computer is a universal computing 
machine, provided that we assume that it has virtually infinite memory. 
For a precise definition see for example j2Qj or We have the following 
theorem 

Theorem 3.1 (Kolmogorov [IS] ). If C and C are universal computing 
machines then 

\AICis, C) - AIC{s, C')\<K (C, C) 
where K {C,C') is a constant which depends only on C and C . 

This theorem implies that the information content AIC of s with respect 
to C depends only on s up to a fixed constant, then its asymptotic behaviour 
does not depend on the choice of C. For this reason from now on we will 
write AIC{s) instead of AIC{s,C). 

The shortest program which gives a string as its output is a sort of en- 
coding of the string, and the information which is necessary to reconstruct 
the string is contained in the program. Unfortunately the coding procedure 
associated to the Algorithmic Information Content cannot be performed by 
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any algorithm. This is a very deep statement and, in some sense, it is equiv- 
alent to the Turing halting problem or to the Godel incompleteness theorem. 
Then the Algorithmic Information Content is a function not computable by 
any algorithm. 

Using the notion of Algorithmic Information Content it is possible to 
define a notion of complexity for infinite strings. Let uj be an infinite string 
on the alphabet A, that is lj € := A^. We denote by the first n 
symbols of the string uj. Then uj"' € A". The complexity measures the 
mean quantity of information in each digit of the string uj. Formally 

Definition 3.2 (Brudno ^8j). The complexity K{uj) of an infinite string 
a; € is given by 

AlCjuj^ 
K [ujj = lim sup 

n^oo 

Using the method of symbolic dynamics it is possible to consider the 
information and the complexity of the orbits of a dynamical system. Let 
{X, /i, /) be an ergodic dynamical system and let Z = {Ii, . . . , /jv} be a finite 
measurable partition of the space X. To the partition Z it is associated the 
finite alphabet A = {1, . . . , N}. Define a map ^pz : X ^ Q, where Q = A^, 

by 

{ipz{x))j = k ^ fix) eh (8) 

The sequence (pz{x) € associated to a point x £ X is called the symbolic 
orbit of X relative to the partition Z. Then we have the following definition. 

Definition 3.3 (Brudno The complexity K{x,Z) relative to the par- 
tition Z of the orbit with initial condition x G X is given by 

AIC{{ipzix)r) 



K{x, Z) := K{(pz{x)) = lim sup 



n 



Remark 3.4. In |Hj and in J2]) using open covers and computable struc- 
tures, notions of complexity of orbits of a dynamical system are defined that 
do not depend on the partition. But in this paper we are only interested in 
the complexity dependent on a partition for the same reason for which we 
study KS entropy relative to a partition. 

This notion of complexity has been related to the notion of KS entropy 
by the following theorem. 

Theorem 3.5 (Brudno ^). Let (X, /i,/) be an ergodic dynamical system 
and Z be a finite measurable partition, then K{x, Z) = h^{f, Z) for ^-almost 
any x € X. 

Theorem 3.6 (White j23| . [24| ). In the same hypothesis as before, for 
^-almost any x £ X it holds 

?i— >oo n 
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If the AIC were a computable function, using the previous theorems 
we could compute the KS entropy relative to a partition. This approach 
can still be useful using optimal compression algorithms, that is algorithms 
which encode symbolic strings, giving an approximation of the information 
contained in a string, hence an approximation of its AIC. For this approach 
see jl], where also a new compression algorithm is presented and applied to 
some well known chaotic systems. 

Remark 3.7. The notion of complexity for dynamical systems has been 
studied for some well known weakly chaotic systems for which it 

holds hfj_{f, Z) = K{x, Z) = for all the partitions, with the aim of giving a 
classification of these systems according to the asymptotic behaviour of the 
AIC of the orbits. 

Remark 3.8. The notion of AIC has been linked to other indices for dy- 
namical systems. For example it has been related to the sensitivity to initial 
conditions {^TT) and to the Poincare recurrence times (0)- 

4 Estimates of the relative KS entropy 

We now introduce formal definitions for random perturbations of a dynami- 
cal system (following |17j). We also give some estimates for the KS entropy 
relative to a partition for the perturbed system using the tool of symbolic 
dynamics. 

Let {wn} be a stochastic process of i.i.d. random variables, with each 
Wn defined on the probability space (VF, "P, P) with values on the interval 
[— cr, cr], and q'^ as the induced distribution. 

Definition 4.1. A random perturbation of the dynamical system {X,iJ,,f) 
is a family of Markov chains on the probability space (W, V, P) with 
values on X and with transition probabilities given by 

P'^ix^B) := P{f:+' eB\f: = x} = q'^{B - f{x)) 

for all Borel sets B C X. 

The meaning of this definition is that the point x is moved to the point 
f{x) under the unperturbed dynamics and then it disperses randomly with 
distribution 

We will consider two different possible actions of the random perturba- 
tion (see [3]). We talk of output noise when the dynamics is driven only 
by the unperturbed map /, and the dispersion is caused by a non exact 
observation of the point /(x). That is given a point x € X, its orbit 
is given by Xn = f{xn-i) and our data are measured observing the orbit 
Vri = fa{xn) = Xn + Wn- Instead we talk of dynamical noise when the dis- 
persion is intrinsic in the dynamics and it is not caused by the observation, 
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in this case the dynamics is driven by So, for a point x € X we have 

Xn = faix) = + Wn- 

Definition 4.2. A probabihty measure fi^ on X is an invariant measure of 
the randomly perturbed dynamical system (X,fu) if 



for all Borel sets B C X. 

We now start from the following assumptions: 

(i) there is an ergodic dynamical system {X, fi, f) with finite KS entropy, 

and denote by /i^(e) its KS entropy relative to a finite measurable 
partition Z of diameter e; 

(ii) we can analyze data produced by a random perturbation (X, fu) of the 

system (output or dynamical noise) with fixed a; 

(iii) there is an invariant and ergodic probability measure on {X, f^) and 
the KS entropy relative to the partition Z for the randomly perturbed 
system is denoted by h^^{e). 

To analyze our system by the Algorithmic Information Content we have 
to use the method of symbolic dynamics. Given the finite measurable parti- 
tion Z = {Ii, . . . , In} of diameter e = we can associate a symbolic orbit 
in Q = {!,..., A^}^ to any orbit in the space X using the map cpz defined 
in equation Q 

In the case of the unperturbed system {X, /i, /) we denote hy Q' C Q the 
image of the map ipz- The set ii' is closed with respect to the action of the 
usual shift map r defined on 17. It is also possible to define the probability 
measure i' on fi' induced by <pz- So we can work on the ergodic dynamical 
system r) with KS entropy h^, = h^{e). 

Analogously we define the map ifz,a from the product space X xW to Q 
in the following way: for each x & X the map ^pz,a{x, •) is the symbolic map 
for the Markov chain So for every point x G X there is a set of possible 
symbolic orbits in := ipz^aiX x W) that correspond to different real- 
izations of the stochastic process {wn}- Let v^j be the probability measure 
induced on fi^ by the measure iia- Then we study the ergodic dynamical 
system {9.'^,v„,t) with KS entropy = V„(e). 

Since now we will denote by ■0 and by u; the symbolic orbits in 17' and 
Vl'^ respectively. 

From the previous definitions, the following propositions easily follow 
Proposition 4.3. Q.' C Q.'^ 

Proposition 4.4. h^{e) < — loge , h^^{e) < — loge . 
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Proposition 4.5. If i^' = and v is absolutely continuous with respect to 
v„ (v «v„) then h^{e) = /i^^(e). 

Proof. For z^cr-almost any a; € it holds 

lim ^ = Ve 9 

n^oo n 

thanks to Theorems 13.51 and 13.61 Hence, since for z^-almost all ^/^ € fi' it 
holds 

ra— >oo n 

and V «Vfj then /i^(e) = h^^{e). □ 

We conclude this section with a lower bound for the e-entropy of the 
random perturbed system. 

Proposition 4.6 (Kifer [17| Theorem 2.4). Suppose that all transition 
probabilities P°'{x,-) have bounded densities p'^{x,y) < K, then h^^{e) > 
— log e — log K. 

Hence the KS entropy of a randomly perturbed system is always infinite, 
since it can be computed as the limit of h^^ (e) for e going to 0. 

5 The AIC for noisy systems 
5.1 The case of output noise 

We now study the behaviour of the AIC relative to a partition in randomly 
perturbed dynamical systems. We start with the case of output noise (see 
previous section) which is easier to be studied and hence it is useful to in- 
troduce the arguments we will use for the more important case of dynamical 
noise. 

Let Q' and Q'^ be the symbolic spaces of the unperturbed and the per- 
turbed systems respectively. We will study the e-entropy of the perturbed 
system using the AIC of the symbolic orbits, ^ and to, of the original system 
and of its perturbed version. 

At any step of our dynamical system the perturbation induced by the 
noise changes the position of the point, and so it could change the set of the 
partition in which the point is. So the n-th symbol of the string uj could be 
different from ipn, the corresponding symbol of the string ip. We recall that 
since we are in the case of output noise, at the step n + 1 there is no memory 
of the noise at the previous step. Moreover since the random variables Wn 
are independent, the probability p of w„ being different from ifjn does not 
depend on the step n, neither on the set of the partition we consider. 



9 



Let H{p) be the KS entropy of a p-BernouUi trial, namely the stochastic 
process of independent BernoulU variables t?„ with parameter p. It is well 
known that 

H{p) = -p log(p) - (1 - p) log(l - p) . 
Hence applying Theorems 13.51 and 13.61 it holds 

lim = H{p) (11) 

n^oo n 

where Hn{p) denotes the average of the AIC over all the sequences produced 
by the p-BernoulIi trial. 

In this framework, we have 

Proposition 5.1. If the random variables Wn have values in the interval 
[—a, a], then 

¥.^MlC{u:^)] < E^iAICir)] + nplog (2 [^j) + H^ip) 

where [•] denotes the upper integer part of a real number and E[-] denotes 
the mean value. 

Proof. To prove the proposition we show an algorithm to describe the 
string a;" and compute the information it needs. Let assume that we know 
the string V"- We have then just to specify the symbols of that are 
different from those of ■0"- We call in this information. We show that for 
all strings cj"" and V'" it holds 

in<rn = {#{i I + w*}) log (2 [^j ) + En[p) (12) 

The algorithm to describe a;" works as follows. First it describes a binary 
string s = (sq, . . . , Sn-i), where Sj = implies that uji = ipi and Sj = 1 
otherwise. To this aim we need on average Hn{p) bits of information, since 
the string s is obtained as a string of a p-Bernoulli trial. 

Moreover the algorithm needs also to specify how far on the left or on 
the right the noise has moved the point. To this aim we need log (2 [f]) 
bits of information for each symbol of different from the corresponding 
symbol of The factor j"^] counts exactly how many sets of the partition 
Z can be covered by the effect of the noise, and the factor 2 is needed to 
specify whether the point is moved to the right or to the left. 

Hence to specify the string we do not need more than AIC^ip") + r„ 
bits of information. If we evaluate the mean of AIC{uj^) over the measure 
//o- then the thesis follows from the definition of the probability p. □ 

Theorem 5.2. If hfj_{e) is the KS entropy relative to a partition Z of diam- 
eter e of an unperturbed dynamical system (X, /), for the e-entropy h^^ (e) 
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of the system perturbed by an output noise {wn} with values in the interval 
[— cj, a] it holds 

<^/.(e)+plog (2 \^])+Hip) 

Proof. The thesis follows from Proposition 15.11 using equations (|10|) and 
© for h^{e) and h^^{e), and using equation ((TT|) . □ 

5.2 The case of dynamical noise 

We now study the case of dynamical noise. We assume to have an un- 
perturbed dynamical system {X, fi, f) and a dynamical perturbation due 
to a stochastic process {wn} of i.i.d. random variables with values in 
the interval [— cj, a] , such that the points x € X follow orbits given by 

Xn+l = f{Xn) + Wn+1- 

Let consider a partition Z of the space X in a finite number of measur- 
able sets {li}, such that the diameter of each set of the partition is e = 1/A^. 
Let Q = {1, . . . , N}^ be the space of the symbolic orbits associated to the 
dynamical system, and denote by w a single string in O^, given by the mea- 
surements made on the system. That is to is the symbolic orbit of a point 
X E X after the effect of the noise. To estimate the Algorithmic Information 
Content of the string uj from above, we show how to construct it from the 
knowledge of the noise and of the unperturbed dynamics. 

Let xq denote the initial point of an orbit of the system. By the clas- 
sical symbolic representation of orbits, there is a symbolic string xp^ = 
('00) V'l) ■ ■ ■ ) £ ^) relative to the partition Z, associated to the point xq 
for the unperturbed dynamical system, that is /'(xq) £ I^o for all i £ N, 
and there is also the symbolic string uj = (loq,uji, . . . ) that is given by the 
perturbed system. We have = V'o- In the case of dynamical noise, we 
have to consider the effect due to the fact that the action of the noise is 
not forgotten at each step. So we have to extend the knowledge of the un- 
perturbed orbit to more than just one iteration. To find a good number 
of iterations to be stored, we use the approximation of the Kolmogorov- 
Sinai entropy Z) = h^{e) relative to the partition Z by the decreasing 
sequence i/(T-"Z|Z„), where Z„ = Z V T-^Z V • • • V T-''+^Z and 

for any two finite partitions P = {Pi) and Q = (Qj). Hence for any 6 > 
there exists no € N such that 

h^{e) - 6 < F(r-"°Z1Z„J < V(e) + 6 (13) 

Let 6 > he fixed, and find the corresponding integer no according to 
equation H13() . Assume that for the initial point xq of the orbit, we know the 
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first no digits of the sequence ip^ . That is we assume to know the cyhnder 
ZriQ^xo), where with this notation we mean the set of the partition Z„q 
containing xq. 

Let xi = f{xo) and xi = xi + wi. As before we denote by tp'^ the 
unperturbed symbohc orbit associated to the point xi. We assume to know 
the action of the noise wi with respect to the partition so we compare 
the strength of the noise a with the diameter^ of the partition 
Moreover, to know Zn^^ixi) we only need the symbol V'no! ^'^'^ know 
once we have we only need the action of w\ with respect 

to the partition Z^q- This is enough to obtain lo\ = V'o- So to obtain the 
first two symbols of the string uj it is enough the following information: 
1(^0,..., l{w^\Zn,) and/^lV'g,...,^-!). We have used /(-I-) 

to denote the conditional information. 

At this point it is possible to iterate the argument with the string 
instead of the string -i/)", hence to find the symbol tJ2 = V'o using the following 
information: l{w2\Zru^^ H'^o^ ■ ■ ■ > V'no-i) (which is given by Zno{xi) and is 
indeed known) and /(t/'iJV'o) • • • , V'^o-i)- 

Iterating this argument we obtain the following proposition: 

Proposition 5.3. If the random variables Wn have values in the interval 
[—a, a], then for any S > there exists no € N and a constant C > such 
that 



E^,[ylIC(u;")]<(n-l) 



h^{e)+d + p\og 2 





a ^ 








)] 









+ Hn-lip)+C 



where [•] denotes the upper integer part of a real number, the probability p 
is defined as in Provosition \5.1\ and = (wQ) • • • > ^n-i)- 

Proof. To obtain the entropy on the right hand side we only have to use 
equation (|13() . and to remark that the conditional entropy H{T~'^° Z\ZnQ) 
is obtained as the integral over the space of all admissible strings of the in- 
formation function I{ipno IV'O) • • • > V'no-i)- The constant C is the information 
I{ipQ, . . . , V'no-i) needed to start the iteration. 

For the part related to the noise we repeat the same argument of Propo- 
sition 15.11 Then the action of the noise can be bounded by 



Ep[rn] = (n-l)plog( 2 
bits of information. 



+ Hn-l{p) 



□ 



Let again /i^^(e) be the e-entropy of the perturbed dynamical system, 
hence it follows 

^In this case we cannot ask the partition Zng to be uniform, so we consider the diameter 
to be the smaller diameter of the intervals that make the partition Zng . 
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Theorem 5.4. Ifh^{e) is the e-entropy of an unperturbed dynamical system 
(X,fj,,f), for the e-entropy hfj_^{e) of the system perturbed by a dynamical 
noise {wn\ it holds: for any 5 > there exists no G N such that 



h^Se)<h^{e)+5 + p\og {l 



+ H{p) 



5.3 Conclusions 



At this point we can make some conclusions about the estimates of the KS 
entropy of a randomly perturbed system. 

We recall the conditions we imposed in Section H] on the data we study. 
So, given a fixed a, we vary the diameter e of the partition considered. 

Let e > a. In this case we can assume that for some e we have Vt' = 
This implies (Propositions 14.^1 and 14. 5^ ) that h^^{e) = h^{e). When $7' C 
we have that, fixed 5 > d such that h^{e) + 5 < — loge, if e„(, > a 

- log e- const < h^^{e) < h^{e) + 5 + p\og2 + H{p) (14) 

using Proposition 14 . 61 and Theorem l5.41 We recall that H{p) takes its maxi- 
mum value log 2 for p = 0.5, and it converges to as p goes to or 1. Then if 
Cno >> o" we have that p converges to 0, hence {p\og 2 + H{p)) converges to 
0, and equation (|T1|) becomes /i/i^(e) < h^{e) + 5. In particular this implies 
equation If instead e^Q < a then we have the same conclusion as in the 
following case. 

If e < 0" then it holds 

— log e — const < h^^ (e) < 

^ / r l^ ^ ^''^ 

<min(^-loge,/i^(e) + (5+plog(^2 ^ j + H{p)j 

using Propositions 14.41 14.61 and Theorem 15.41 Again if e„y « a then p 
converges to 1, hence H{p) converges to 0, and the estimate on the right 
hand side of equation becomes h^{e) + 6 — log e„(, + const. 

From equations (|14j) and (|15|1 it emerges that by computing the KS 
entropy relative to partitions, we obtain an indication of the smallness of 
the random perturbation present in the data we are studying. Indeed if in 
the curve h^^{e) we find an almost flat region (e > ei), and as e decreases 
there is a change in the behaviour of the curve, that almost behaves as 
— log(e) (e < £2), then the interval (e2,ei) should be a good approximation 
for the size a of the perturbation. In the next section we apply this method 
to a perturbed dynamical system, using a compression algorithm to estimate 
the Algorithmic Information Content. 

Moreover this method can also be thought of as a method to distinguish 
between purely stochastic systems and randomly perturbed dynamical sys- 
tems. Indeed for randomly perturbed systems equations (fTl|) and (|15j) sug- 
gest the presence of a change of behaviour for the e-entropy at the level 
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of the perturbation. This change should not appear in purely stochastic 
systems. However this aspect has to be examined further. 

6 Numerical experiments 

In the previous section we have proved that the Algorithmic Information 
Content can be used for randomly perturbed dynamical systems to obtain 
information on the size of the random perturbation. We remark that unfor- 
tunately the AIC, as defined in equation Q, is not a computable function. 
So to compute the information content of a string one needs to use a com- 
pression algorithm, that is an algorithm which approximates the value of 
the information function AIC. 

Formally, a compression algorithm is defined as a recursive reversible 
coding procedure Z : A* ^ {0, 1}* (for example, the data compression 
algorithms that are in any personal computer). The information content 
Iz of a finite string s computed by the algorithm Z is given by the binary 
length of the compressed string, that is Izis) = \Z(s)\. 

Not all compression algorithms approximate the AIC. In Slj this point is 
discussed in details. For the purposes of this paper, it is sufficient that 

limsup = K[uj) 

n— >oo IT' 

where K{u;) is the complexity of an infinite string uj (see Definition 13.2)1 . 
The algorithms with this property are called optimal. 

In [3] it is also introduced a compression algorithm called CASToRe, 
which has been created with the specific aim of studying weakly chaotic 
dynamical systems, namely chaotic systems with null KS entropy. 

Using CASToRe we have analyzed an example of randomly perturbed 
dynamical system. We have chosen the logistic map 

f : X Ax(l — x) 

on the interval [0, 1] with A = 4. To this map we have added a dynamical 
noise given by independent random variables {wn} uniformly distributed on 
the interval [—(7,0"]. 

Keeping a fixed we have varied the diameter of the partition from 0.5 to 
0.004=1/250, and we have computed the complexity of the 10^-long symbolic 
orbits of the perturbed system relative to the different partitions. 

In figure n we show the results. It is a log-linear plot of the complexity 
of the symbolic orbits versus the diameter of the partition, plotted on the 
X axis. The solid line is the upper bound — log e, and the other curves are 
the empirical curves which correspond to the following values of a: 0.5, 0.1, 
0.02, 0.01, 0.001 from the upper to the lower, respectively. For each of the 
curves h^^{e) it is possible to identify the intervals (e2,ei), that give good 
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Diameter of the parlitioa 

Figure 1: The complexity for different values of a with respect to the diameter 
of the partition. 

approximation of o", in agreement with the theoretical result in equations 
(|TH) and (P3|) and the subsequent comments. 

When the diameter of the partition is high with respect to the size of the 
noise, then we have a good approximation of the unperturbed KS entropy of 
the system, since the partition with diameter 0.5 is generating. In figure ^ 
it is possible to see that the complexity of the symbolic orbits for partitions 
with diameter close to 0.5 is close to 1 (the KS entropy^ of the logistic map) 
for the empirical curves with a < 0.1. 
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